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This report describes the Human System Interface (HSI) analysis, design and test 
activities that will be performed to support the development of requirements and design 
guidelines to facilitate the incorporation of High Altitude Long Endurance (HALE) 
Remotely Operated Aircraft (ROA) at or above FL400 in the National Airspace System 
(NAS). These activities are required to support the design and development of safe, 
effective and reliable ROA operator and ATC interfaces. This plan focuses on the 
activities to be completed for Step 1 of the ACCESS 5 program. Updates to this 
document will be made for each of the four ACCESS 5 program steps. 
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1 INTRODUCTION 


1.1 Overview 

This plan describes the Human System Integration (HSI) activities that will be 
performed to support the development of requirements and design guidelines to 
establish certification guidelines/requirements for High Altitude Long Endurance 
(HALE) Remotely Operated Aircraft (ROA) to fly at or above FL400 in the 
National Airspace (NAS). The intent is to provide file and fly access to the NAS 
for HALE ROAs, identically to that provided to manned aircraft. This plan 
focuses on the tasks to be completed for Step 1 of the ACCESS 5 program. 
Updated plans for Steps 2 through 4 will be developed prior to the 
commencement of each step. 

The ACCESS 5 HSI IPT will perform applicable MIL-HDBK-46855B analyses, 
design support and test activities to support requirements development and to 
establish HALE Air Vehicle Control Station (AVCS) functional requirements and 
design guidelines. In addition, specific procedures and design guidelines will be 
developed in support of the Simulation IPT so it may develop a generic AVCS for 
test and evaluation. Furthermore, the task will evaluate the impact of ROA 
operations on the air traffic control system. Activities performed in Step 1 will 
not address HSI requirements for ROA maintenance personnel. These 
requirements may be addressed in subsequent steps of this effort. 

Major activities to be accomplished in Step 1 will include: 

a. Literature Review - Reviewing applicable FAA standards, FARs, FAA 
Order 7110.65P, and guidance provided in the Aeronautical Information 
Manual, etc. to support the development of AVCS design requirements, and 
pilot/operator knowledge skill, and ability (KSAs) requirements; and the 
impact of ROAs on air traffic control. 

b. ROA and ROA Pilot Analyses - Performing analyses to determine pilot 
functional requirements, and generic ROA pilot knowledge, skill and ability 
requirements. Analyses will include Mission, Function, and Task and 
Workload to identify high level requirements to enable the ROA pilot to 
comply with relevant, proposed, FAA ROA pilot and Air Vehicle Control 
Station (AVCS) certification criteria. 

c. Supporting the development of Pilot Rating Criteria for HALE ROA - 

Results from Mission, Function and Information and Control analyses will 
be used to derive preliminary pilot rating criteria. 

d. ROA Functional Capability and Compliance - Identifying functional 
areas where ROA capabilities cannot satisfy aircraft regulatory and air 
traffic control requirements. 

e. Simulation and Flight Test IPT Support - Providing an input into the 
Simulation and Flight Test IPTs on the design of the AVCS, and on the 
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mission scenarios/test events that will be used to conduct these 
simulations/flight test activities. 

f. Air Traffic Control Analyses - Performing analyses to the potential impact 
of ROA operations on ATC operator task requirements/workload. The goal 
is for the ROA system to be transparent to ATC. However, if it is 
determined that ATC operations will be impacted, then design guidelines 
will be developed to minimize the impact of ROA operations on the ATC 
system. Analyses may include Function, Information, and Control Analyses 
to identify the ability of the controller to comply with relevant, proposed, 
FAA requirements. 

The results of these activities will be used to develop preliminary HSI design 
guidelines and AVCS functional requirements. As part of this process, the HSI 
IPT will develop comprehensive human factors requirements, and will provide 
guidance on methods for complying with proposed FAA requirements. The intent 
is not to develop a specific point design, but to develop generic, ROA guidelines 
that can be used by ROA manufacturers to obtain FAA approval for Experimental 
Certification. The infonnation provided will also be of value to FAA personnel 
who are assigned the responsibility for AVCS certification. In addition, this effort 
will provide guidance to the Simulation and Flight Test IPTs in the form of 
specific HSI design recommendations to support simulation and test activities. 


1.2 Report Organization 

This plan describes the tasks to be accomplished, the ACCESS 5 HSI IPT 
Organizational Structure, program interactions, and provides a schedule for Step 1 
task accomplishment. 
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2 ORGANIZATION 


2.1 ORGANIZATIONAL STRUCTURE 

The ACCESS 5 organizational structure is shown in Figure 2-1. This IPT 
structure provides for a close interaction of HSI personnel with all other IPTs. 
HSI personnel will coordinate their activities with all other IPTs (i.e., Policy, 
ROA Impact, Technology, Simulation and Flight Test) to ensure that HSI 
considerations are included as required in the performance of these work 
packages and to minimize duplication of effort. 
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Figure 2-1: HSI Organizational Structure 

2.2 ACCESS 5 IPT RELATIONSHIPS 

To ensure that HSI requirements are coordinated with the other IPTs, HSI 
personnel will accomplish the following activities: 

a. Review all IPT work packages to determine what HSI inputs are required 
and to minimize duplication of effort 

b. Participate in all major work package program reviews 

c. Conduct Technical Interchange Meetings (TIMs) with other IPTs to 
facilitate coordination and work package deliverables. 
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2.3 ACCESS 5 HSI IPT Key Personnel 


The following table provides a list of key HSI personnel, their affiliation, applicable 
experience and their roles on the HSI IPT. 


Table 2-1: HSI Personnel 


Nam 

Company 

Applicable Experience 

Access 5 Role 

Sally Moore 

The Boeing 
Company 

20+ years human factors 
experience; pilot 

HSI Work Package Lead 

Gary Gershzohn 

The Boeing 
Company 

Airline pilot; air traffic 
controller; human performance 
expert; accident investigator 

Human Factors support, and 
SME for ROA Pilot and 
ATC operations 

Barry Berson 

Lockheed 

Martin 

30 + years of applicable Human 
Factors experience 

Human Factors Support 

Laura Boltz 

Lockheed 

Martin 

10+ years of applicable Human 
Factors experience 

Human Factors Support 

Ray McAdaragh 

FAA/NASA 

25 + years of applicable Human 
Factors experience 

Human Factors Support 

Mike Schultz 

MTSI 

15+ years Collision Avoidance 
and C3 experience/HSI 

Cooperative Conflict 
Avoidance Technology 

Ken Zugel 

Aurora Flight 
Sciences 

20+ years experience HALE 
ROA Pilot 

ROA Pilot 

Wayne Sadler 

AeroVironment 

10+ years experience HALE 
ROA Pilot 

ROA Pilot sme 

Bernie Schmidt 

Northrop 

Grumman 

20+ years pilot and systems 
engineering 

ROA Pilot SME 

Bob Blair 

General 

Atomics 

18+ years pilot, test pilot 

Pilot SME 

Richard Zanecki 

General 

Atomics 

20+ years pilot 

ROA Pilot 

Craig Bomben 

NASA 

20+ years pilot 

Pilot 
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3 HUMAN SYSTEMS INTEGRATION IN SYSTEMS ANALYSIS 

3.1 OVERVIEW 

This section summarizes the analyses that ACCESS 5 HSI IPT personnel will 
perform to define AVCS pilot crew rating requirements, and to develop AVCS 
design guidelines and to assess the impact of HALE ROAs on ATC procedures 
and HSI design impacts. The results of these activities will support the 
development of Experimental Certification requirements for HALE ROAs. 

3.2 TECHNICAL APPROACH 

A description of the HSI analyses that will be performed to develop Experimental 
Certification Requirements and design guidelines for the integration of HALE 
ROAs into the NAS is contained in this section. This effort will focus on defining 
HSI requirements for the ROA pilot and air traffic controller. Requirements for 
Mission Planners, Sensor Operators, Maintainers, etc. are out of scope for this 
effort, and therefore will not be included in the Step 1 HSI Program Plan. 

A series of Pilot and ATC controller reviews will be conducted to accomplish the 
work described in this section. The first Pilot review will concentrate on 
obtaining data to accomplish the mission and function analyses described in this 
section. Subsequent reviews will be conducted to perform the technology trade 
studies and to accomplish the Information and Control requirements analysis. 

It is planned to conduct an ATC controller review to identify any potential HSI 
impacts resulting from the integration of HALE ROAs into the NAS. The 
objective of this review will be to identify potential impacts, and to obtain Subject 
Matter Expert (SME) opinions on how to reduce/eliminate any potential impact. 

Figure 3-1 provides a Functional Flow Block Diagram of our approach for 
performing the required HSI analyses. This block diagram also illustrates the 
design tasks that are to be accomplished in Step 1 . The design tasks are described 
in the next section. A brief description of our proposed analysis methodology is 
provided below. 

3.2.1 Identify Preliminary HSI Requirements 

This task consists of reviewing all relevant FAA documents that might impose 
requirements that would impact the certification of a HALE ROA AVCS. A preliminary 
list of relevant FAA Standards, Technical Orders, FARs is provided below. 

• FAR Part 23/25 

• FAR Part 91 

• Aeronautical Information Manual 

• FAA Order 71 10.65P 
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• FAA Human Factors Design Standard 

The output of this activity will be a detailed checklist. The checklist will contain 
sections that summarize all applicable requirements that relate to: 


Functions and tasks required of the pilot 

Functional requirements for ROA and AVCS hardware, software, and 
procedures 

Crew Rating requirements, and crew duty cycles 
Design of the AVCS HSI 
ATC Functions and Tasks 


Proposed regulatory requirements for ROA operations in the NAS (e.g., 
FARs) 

Proposed changes to FAA Order 71 10.65 

Guidance on methods for manufacturer’s compliance with proposed 
requirements 


Identify Preliminary 
HSI Requirements 

-> 

Perform Preliminary 
HSI Analyses 

- 

Perform HSI 
Tech Assessments 





Communications 
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Figure 3-1: HSI Analysis Methodology 


3.2.2 Perform Mission and Function Analyses 
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3.2.2. 1 Mission Analysis 

This step consists of defining and decomposing a generic HALE ROA mission. 

To obtain this data, experienced ROA pilots and HSI personnel will work together 
to perfonn Mission and Function Analyses for ROA operations. The initial 
activity will concentrate on ROA operations above 40,000 feet. In subsequent 
Steps of the program, analyses will be conducted to concentrate on all other 
mission phases (Taxi, Takeoff, Climb, Cruise, Enroute operations above 18,000 
feet, Approach and Landing). The data form provided in Appendix A will be 
used to collect the required data. 

Missions will be decomposed to identify: 

a. Mission Events. Significant nominal and contingency events relevant to 
operations at or above 40,000 feet. 

b. Operator Tasks. Sequential list of all tasks that ROA pilot has to perfonn 
to accomplish mission objectives for flight at or above FL400. 

c. Information Requirements. Preliminary listing of the information 
required by the pilots to accomplish identified tasks. 

d. Control Requirements. Listing of the control requirements to accomplish 
each task. 

3.2.2.2 Mission Decomposition 

A mission profile will also be developed to provide a generic graphical 
representation of the vehicle’s trajectory as a function of mission phase/segment. 
Mission segmentation consists of decomposing a mission into the major 
sequential activities that occur from the beginning to the end of the mission. At 
this stage of development, a three-level structure is being used to decompose the 
mission (i.e., Phases, Segments and Subsegments). These levels are defined as 
specific time periods within the mission which are grouped on the basis of 
homogeneity of operations. These levels have identifiable start and stop points 
(e.g., for the Takeoff Phase the start point is defined as the time when power is 
applied to the vehicle and ends when the vehicle lifts off the ground). 

The purpose of dividing the mission into segments is to be able to analyze 
smaller, more convenient units of behavior. Mission decomposition includes: 

a. Naming the mission phase/segment/subsegment 

b. Defining the major roles and responsibilities for the pilots within each 
time period 

c. Deriving start, stop and duration times for each mission phase, 
segment, and subsegment 

A narrative summary of the major actions/events that occur within each mission 
time period will be developed to provide a representative structure for 
decomposing a generic HALE ROA missions. For Step 1, this analysis will be 
limited to the enroute portion of a flight (at or above FL 400). 
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3.2.2.3 Function Analysis 

This activity consists of identifying the major functions that the vehicle systems 
and aircrew must perform to accomplish mission objectives. In effect, what the 
vehicle systems and operators have to accomplish during the mission define the 
major system functions (e.g., Flight Operations, Navigation, Communication, 
etc.). 

These primary functions are then decomposed to identify: 

a. Subfunctions (e.g., for the communication function, sub-functions include 
short, and long range communications, etc.). 

b. Systems (e.g., the short range communication subfunction might include a 
V/UHF and Satellite communication systems, etc.). 

c. System Functional Requirements (e.g., the UHF radio system includes; a 
transmitter, a receiver, associated antennas, system controls and indicators, 
etc.). 

d. Preliminary Control and Information Requirements (e.g., to control a 
radio, control functions must enable the operator to turn the system on, to 
transmit and receive messages, etc. In addition, the system must provide 
specific information to enable the operator to use the system effectively 
(e.g., channel and frequency, current operational and available modes, 
system status, etc.). 

This functional breakdown is required to identify the major tasks/jobs that the 
systems and pilot/ATC must perform to accomplish system objectives. At this 
stage of the analysis process, the functional breakdown is independent of the 
mechanisms used to accomplish a given function/task (i.e., whether a given 
function is allocated to a specific operator, to hardware/software, or to a 
combination of the two). 

3.2.2.4 Function Allocation 

In this step, the system functions are allocated to hardware/software, human 
operator, or to a combination of the above, based upon mission performance 
requirements, and on relative human and machine capabilities and limitations. 
Preliminary allocations will be based mainly on analysis results, and on the 
subjective perceptions of experienced operational, human factors and engineering 
personnel. These preliminary allocations will be refined based the results of 
simulation and flight-testing. Other allocation factors include; allocating 
functions based upon historical precedent (i.e., how functions are currently 
allocated on inhabited aircraft and ROAs) and relevant psychological literature 
describing human capabilities and limitations. 

3.2.3 Perform HSI Technology Assessments 

In this step, work will be performed in support of simulation and flight test 
activities. To develop an AVCS for simulation purposes, trade studies will be 
accomplished to identify and evaluate current HSI procedures and control and 
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display interfaces used in current day HALE ROA AVCSs. The goal of this 
effort is to identify best practices for the integration of required technologies into 
the simulator AVCS. Based on test and evaluation results, this information will be 
used to develop preliminary HSI design guidelines. For Step 1, trade studies will 
be performed on communications (LOS and BLOS), Contingency Management, 
Cooperative Conflict Avoidance, and Weather Information systems. 

To perform the trade studies, the following tasks will be accomplished: 

• Identify Alternatives 

• Develop Assessment Criteria (Quantitative and Qualitative) 

• Weight Criteria, if applicable 

• Identify Subject Matter Experts to perform the evaluations 

• Tabulate results and identify high potential alternative(s) 

The results of Mission, Function and Trade study analyses will be used to develop 
preliminary control and display guidelines for integrating these technologies into 
the simulator AVCS. This activity will include determining: 

a. Information required by the ROA pilot to perform assigned tasks quickly 
and accurately. This includes the determination of cues, alerts, graphical 
and tabular information required to support task performance. 

b. Determining controls required to support task execution. This will include 
determining the criticality, frequency and sequence of use of required 
control devices. This information will be used to provide 
recommendations/design guidance for arranging controls and displays 
within the simulator AVCS. 

c. Develop preliminary concepts for the content and format of the Primary and 
Auxiliary Displays, including concepts for information coding (e.g., shape, 
color, line thickness and type, etc.) prioritization and display decluttering. 

The results of the trade studies will be used to derive high potential HSI concepts 
that will be integrated into the AVCS that will be used to perform the Simulation 
IPT part-task and full mission simulations. 


3.2.4 Task and Workload Analysis 
3.2.4. 1 Task Analysis 
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Typically, in order to perform a task analysis, a preliminary workstation design is 
required. For this analysis, the results from the previous analyses and trade 
studies will be used to identify generic tasks that have to be performed by current 
day ROA pilots in the conduct of missions. For Step 1, this analysis will focus on 
tasks that are performed for the transit mission phases at or above 40,000 feet. 
Pilot SMEs, working on the HSI Work Package, and data obtained from 
simulation and flight test activities will be used to identify normal and 
contingency tasks for this mission phase as well as provide input into the 
assessment of the behavioral requirements for the tasks identified. 

To support the development of AVCS Crew Rating requirements, the task 
analysis will identify and evaluate pilot skill, knowledge, and ability 
requirements. This preliminary task analysis will be performed to assess the: 

a. Information presented to inform the AVCS pilot that a given task is to be 
performed, 

b. Decisions the pilot has to make to perform a given task, 

c. Controls and displays used by the pilot to accomplish a given task, 

d. Specific control actions performed, and 

e. Information provided (feedback) to enable the pilot to determine the 
adequacy of task perfonnance. (In most cases, the feedback provided to the 
operator for performing one task becomes the indicator for performing the 
next task). 

In addition, analyses will be conducted to determine the potential impact on ATC 
operations from the integration of ROAs into the NAS. To obtain this data, the 
HSI IPT will work closely with the ROA Impact and Simulation IPTs. In 
addition, data obtained from flight demonstrations will be used to support this 
analysis. If potential impacts are identified, the HSI IPT will perform requisite 
analysis to develop preliminary procedural, control and display concepts for 
minimizing the impact on ROA integration on controller workload, and to 
enhance SA. Promising concepts will be integrated into the Simulation IPT 
studies to assess their effectiveness. 

The task analysis will provide data for: 

a. Assessing the relative effectiveness of current function allocation schemes, 

b. Identifying functional areas where ROA capabilities cannot satisfy aircraft 
regulatory and ATC requirements 

c. Establishing standards and conventions to avoid potential HSI problems 

d. Identifying Critical Tasks for more detailed analysis. 

e. Establishing preliminary AVCS Pilot Rating requirements 
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3.2.4.2 Workload Analysis 

To assess workload, two metrics will be derived (1) Activity Level, and (2) Task 
Difficulty. Where Activity Level is defined as the ratio of the time required to 
accomplish given tasks divided by the time available for task accomplishment. 
To assess Task Difficulty, the McCracken- Aldridge Workload Scales will be 
used. Separate scales are provided to assess the auditory, visual, cognitive and 
psychomotor resource demands of crew allocated tasks. Seven-point anchored 
rating scales are provided for each scale. On these scales, a 1 is used to indicate 
very low task demands, and a 7 is used to indicate very high or excessive 
demands. Figure 3-2 shows the McCracken-Aldridge scale for Cognition. 

Table 3-1: McCracken-Aldridge Cognitive Workload Scale 


Scale Value 

Descriptor 

1.0 

Automatic (Simple Association) 

1.2 

Alternative Selection 

3.7 

Sign/Signal Recognition 

4.6 

Evaluation / Judgment (Simple) 

5.3 

Evaluation / Decoding (Recall) 

6.8 

Evaluation / Judgment (Complex) 

7.0 

Estimation, Calculation or Conversion 


To support the identification of high workload time periods within the mission, 
multiple criteria will be used (e.g., mission time period where an operator has an 
average Activity Level of 80 % or greater, and average task demands, for any one 
resource greater than 5.5 on the McCracken-Aldridge Scales). 

Workload analysis results will provide data for identifying: 

a. Mission "Choke-points" (Mission time periods where operators 
experience excessively high workload levels, (i.e., activity levels 
greater than 80%, or resource demand levels greater than 5 on any 
resource dimension)); 

b. Operator task overload, either due to excessive Activity Levels or Task 
Demands; and 

c. Candidate tasks for redesign of the simulator AVCS. Options for 
redesign include: 

(1) Simplifying the design of the controls and displays used to 
accomplish the task, 
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(2) Automating the task or providing a decision aid to facilitate task 
execution, or 

(3) Reallocating the task to a "less busy"/ “less stressed” operator. 
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4 HUMAN SYSTEMS INTEGRATION IN DESIGN 

4.1 OVERVIEW 

This section summarizes the major design activities that will be accomplished in 
Step 1 to support the development of Experimental Certification Requirements, 
and preliminary AVCS and ATC guidelines. 

4.2 HSI DESIGN ACTIVITIES 

HSI personnel will perform the following tasks to achieve the objectives specified 
above: 

4.2.1 Develop Preliminary HSI Design Guidelines 

This task will generate a set of preliminary HSI design guidelines that will be 
used to support the development of the AVCS that will be used to conduct the 
part task and full mission simulations. 

To develop these design guidelines the following activities will be performed. 

a. Literature Review - Reviewing applicable FAA standards, FARs, FAA 
Order 7110.65P, and guidance provided in the Aeronautical Information 
Manual, etc. to support the development of AVCS design and pilot/operator 
skill, ability and knowledge requirements; and the impact on air traffic 
control. Starting with the FAA Human Factors Design Standard 
(DOT/FAA/CT-96/1, design guidance from Human Factors specifications 
and standards (e.g., MIF-STD-1472, JSSG 2010, etc.), and from Graphic 
User Interface (GUI) standards, and previous design guidelines (e.g., 
Unmanned Aerial Vehicles Human Factors Guidelines (Richardson and 
Gibson, 1998), etc.) will all be reviewed and integrated into our preliminary 
Step 1 HSI Design Guide. 

b. Supporting the development of Pilot Rating Criteria for HALE ROA - 

Results from analyses and reviews with subject matter experts will be used 
to derive preliminary Pilot rating criteria 

c. ROA Functional Capability and Compliance - Identifying functional 
areas where ROA capabilities cannot satisfy aircraft regulatory and air 
traffic control requirements. 

d. Simulation and Flight Test IPT Support - Providing an input into the 
Simulation and Flight Test IPTs on the design of the AVCS-that will be used 
to perform part-task, full-mission simulations, and flight tests, and on the 
mission scenarios/test events that will be used to conduct these 
simulations/flight test activities. 

e. Site Visits - Conducting site visits and interviewing ROA pilots to identify 
what they like/do not like about current day AVCSs. Obtain lessons learned 
data on problems/difficulties experienced in present day operations. 
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f. Mishap/Accident Data Review- Reviewing mishap/accident data to 
determine whether the ROA pilot, or the design of the AVCS was listed as a 
contributing factor in ROA/UAV accidents. 

g. Simulation and Flight Test - Supporting Simulation and Flight Test 
activities to observe AVCS operations, and to obtain objective and 
subjective ROA pilot and ATC Controller performance and subjective 
preference data. 

Topics to be included in the HSI Design Guide will include: 

• Visual displays - Analog, digital, qualitative, pictorial and symbolic, static 
and dynamic graphics 

• Auditory displays - Verbal and non-verbal 

• Integrated Caution and Warning Systems - Visual and auditory components 

• Kinesthetic and tactile displays - Determining control position and load 
through amount of movement and resistance 

• Controls - Rotary (knobs, cranks, wheels), linear (levers, push-buttons, 
pedals, handles), and special purpose (remote handlers, manipulators) 

• Specialized input devices - Keyboards, and computer interfaces 

• Control-display dynamics - Movement ratios, compatibility, aided controls, 
quickened displays 

• Physical environmental factors affecting human performance - Lighting, 
noise, vibration, motion, atmospheric composition and pressure, and 
temperature 

• Workspace layout - Limits of working area, physical and visual access, 
orientation, location and arrangement of components 

4.2.2 Establish HSI Experimental Certification Basis and Compliance 
Methodology 

This activity will focus on developing preliminary human factors requirements, 
derived from FARs and other FAA documentation, to provide a certification basis 
for obtaining an Experimental Certification, and will describe methods of 
compliance for verifying preliminary requirements. The FAAs Policy Statement 
Number ANM-99-2 (Requirements for Reviewing Certification Plans to Address 
Human Factors for Certification of transport Airplane Flight Decks) will be 
tailored to derive preliminary certification and compliance methods for the 
AVCS. The content and format of this document is described in the Deliverables 
section of this report. 
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5 


HUMAN SYSTEMS INTEGRATION IN THE DERIVATION OF 
PILOT RATING REQUIREMENTS 


5.1 OVERVIEW 

HSI IPT personnel will play a major role in the development of proposed pilot 
rating criteria. (Maintainer requirements may be addressed in subsequent steps of 
the ACCESS 5 program.) Specifically, information contained in the Task 
Analyses will indicate the decisions that these personnel will be required to make 
to accomplish assigned tasks. Identification of these decision-making 
requirements (i.e., what information the pilot needs and what decisions he/she has 
to make) provides essential information for determining skill, knowledge and 
ability requirements. 

As indicated above, pilot-rating requirements will be derived from the results of 
HE analysis, design and evaluation activities. Task and workload analyses will be 
conducted to define and evaluate generic operator information, control and 
display requirements to monitor and control ROAs in the NAS for mission 
operations at or above FL 400. In addition, the results of Simulation and Flight 
Test evaluations will be used to update and refine the analysis data. 
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6 HUMAN SYSTEMS INTEGRATION IN TEST AND EVALUATION 


6.1 OVERVIEW 

HSI personnel will perform detailed evaluations to support the development of HALE 
ROA HSI requirements. Part task, full mission and flight test evaluations will be 
conducted. The HSI test program will be integrated within the overall Simulation and 
Flight Test IPT programs to ensure completeness of the data collection activities and to 
minimize duplication. Figure 6-1 summarizes the major HSI study areas, describes the 
purpose of each evaluation, how the evaluation is to be accomplished and applicable 
performance measures. 

Table 6-1: Major HSI Study Areas for Test and Evaluation 


STUDY area 

OBJECTIVES 

ISSUES 

METHODS 

PERFORMANCE 

MEASURES 

1. avcs 

Functional 

Design 

Capability and 
Operability 

Functional Design 
Compliance to 
applicable 
FARs /Design 
Standards 

Workload 
Management and 
SA 

• Information 
availability 

• Control 
Capabilities 

• Operability, 
Workload and 
SA 

• Analyses 

• Part Task and 
Full Mission 
Simulation 

• Flight Test 
Evaluations 

• Compliance with 
applicable FAA/HSI 
Standards and 
Specifications 

• Empirical 
Performance and 
Subjective Opinion 
Data 

2. ATC 
Procedures/ 

Minimize Impact 
of ROA operations 
on ATC/NAS 

• ATC Workload 
and SA 

• ATC Procedural/ 
Documentation 
Review 

• Analyses 

• Specification 
Compliance 
Checklists 

• Part Task and 
Full Mission 
Simulation 

• Flight Test 
Evaluations 

• Compliance with 
applicable FAA/HSI 
Standards and 
Specifications 

• Empirical 
Performance and 
Subjective Opinion 
Data 

3 ROA Pilot 
Controls and 
Displays 

Develop 

Requirements and 
Design Guidelines 

• Control and 
Display design 
based on HSI 
Analysis, Design 
& Test Results 

• Effective use of 
Multicolor 
displays and 
integrated 
controls. 

• Ease of use 

• Reduce 
Workload, 
Personnel and 
Training 
Requirements. 

• Design 
Checklists 

• Subjective 
Opinion Surveys 

• Derivation of 
Information 
Display and 
Control 
Requirements 

• Mockup 
Evaluations/ 
Operator 
Surveys 

• Ground and Test 
Evaluations 

• Empirical system and 
Aircrew/Main tainer 
Performance Times 

• Subjective and 
Empirical data on 
Aircrew/Main tainer 
Performance and 
Workload Levels 
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STUDY AREA 

OBJECTIVES 

ISSUES 

METHODS 

PERFORMANCE 

MEASURES 

4 Pilot Rating 
Criteria 

Develop 

preliminary Pilot 
Rating Criteria 

• Pilot skill, 
knowledge and 
ability 

requirements 

• Physical 
Capabilities/ 
Limitations 

• Preliminary 
Training 
Requirements 

• HSI Analyses 

• Subjective 
Opinion Surveys 

• Part and Full 
Mission 
Simulations and 
Flight Test 
Evaluations 

• Subjective and 
Empirical data on 
Aircrew/Main tainer 
Performance and 
Workload Levels 


6.2 HUMAN SYSTEMS INTEGRATION EVALUATION ACTIVITIES 

6.2.1 Part Task and Full Mission Simulation 

Part task and full mission evaluations will be performed to support the 
development of requirements, design guidelines and to assess the utility of 
selected HSI concepts on AVCS pilot and ATC performance. HSI personnel will 
develop and administer questionnaires to obtain ROA pilot and air traffic 
controller preference data. In addition, objective performance data (time to 
complete specified tasks, errors made, etc,) will be obtained. These evaluations 
will also assess promising HSI concepts for integrating key Step 1 technologies 
(Communications, Contingency Management, Cooperative Conflict Avoidance, 
Weather information, etc.). Again, the goal here is to develop requirements/ 
design guidelines, and not to develop a point design to be imposed on ROA 
manufacturers. 

Specific areas to be addressed in the HSI test and evaluation effort will include: 

a. ROA Pilot Functions and Tasks - Obtain data to support the development 
of Pilot skill, knowledge and ability requirements 

b. AVCS Control and Display Designs. Issues to be addressed will include: 

1 . Information interfaces and aircrew station signals, including symbol 
and character sizes, information density and readability, 
understandability of symbology, spatial orientation, etc., will be 
addressed to support requirements derivation and to support the 
development of design guidelines 

2. Aircraft system control and data entry. This includes determining 
effective concepts that will enable the AC VS Pilot and ATC operator 
to interact with their systems to reduce workload and facilitate SA and 
safety. 

3. Performing assessments to verify that AVCS HIS functional designs 
satisfy aircraft regulatory and ATC requirements 

c. Workload. Results of these evaluations will be used to determine whether 
additional design refinements (HSI design simplification, function 
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automation, requirements for decision aiding, reallocating task to the other 
operator, etc.) are required to reduce operator workload levels. 

d. AVCS New Technology Integration. The assessment of the impact of new 
technology integration on operator performance and mission effectiveness 
will be included in the control-display and workload evaluations described 
above. Full consideration will be given to assessing the impact of new 
technologies on pilot skill, knowledge and ability requirements. 

e. Air Traffic Control. Test and evaluation will be performed to assess the 
impact or ROA operations on ATC controller workload and SA. 


6.2.2 Flight Test Evaluations 

The HSI activities performed in support of Flight Test evaluations will provide 
data that will support the definition of Pilot KSAs. In support of these objectives, 
HSI personnel will support the following T&E activities: 

HSI personnel will develop and administer questionnaires to obtain ROA pilot 
and ATC operator preference data. In addition, objective performance data (time 
to complete specified tasks, errors made, etc,) will be obtained to support the 
assessment of significant ROA pilot, AVCS, and ATC issues. These may include 
topics such as: 

1. Adequacy of the design and operability of the controls and displays. 
This also includes assessing concepts for function automation, 
adaptive systems and for decision aids, as well as assessing the 
utility of new technology integration, 

2. Acceptability of workload levels and SA 

3. Potential impact of ROA operations on ATC controllers and on the 
NAS 

The results of all of these activities will be documented in the applicable 
ACCESS 5 Test Reports. HSI design deficiencies and problem sheets will be 
documented and discussed with ACCESS 5 personnel to derive solutions to 
identified problems. 
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7 HUMAN SYSTEMS INTEGRATION DELIVERABLE DATA 
PRODUCTS, SCHEDULE AND MANPOWER REQUIREMENTS 

7. 1 HSI Data Products 

The following documents will be provided to summarize the procedures 
employed and the results obtained from the HSI IPT analyses, design and test and 
evaluation activities accomplished during Step 1 . Deliverables for 2004 through 
2006 are provided below. 

7.1.1 2004 HSI Deliverables 

a. Step 1 HSI Program Plan. This plan briefly describes the tasks that are to 
be accomplished by the HSI IPT in support of the ACCESS 5 program for 
the current step. It is planned to update the HSIPP at the beginning of each 
step. 

b. Preliminary HSI Concept Requirements Definition. This preliminary 
report will serve as a repository for all information obtained in 2004 
relating to the interface between the ROA pilot and ATC. Information 
from scheduled workshops, literature reviews, and observations from the 
Alaska Flight Test program will be used to generate the information 
contained in this report. 

c. Preliminary HSI Design Guide. This preliminary report will serve as the 
repository for all HSI developed, FAA approved requirements, guidelines 
and procedures as applied to the HSI aspects of the ACCESS 5 program. 


7.1.2 2005 HSI Deliverables 

a. Initial HSI Functional Requirements for ROA Flight in the NAS (9/05). 

This activity consists of identifying the major functions that the ROA vehicle 
systems and aircrew must perform to accomplish mission objectives. The report 
will identify primary functions and decomposed elements in the following 
categories: Crew Subfunctions, System Functional Requirements, and 
Preliminary Control and Information Requirements. 

b. Initial HSI Requirements and Guidelines for Experimental Certification 
(Draft - 3/05 Step 1 Final - 9/05). This document will provide requirements for 
the certification of a ROA AVCS. This document will describe methodologies 
that can be used to achieve AVCS certification. This report will also describe 
certification criteria, issues and validation concepts. It is planned to update this 
report at the end of each step. 

c. Initial HSI Requirements for Pilot Rating (Draft 3/05, Final 9/05). This 
report will describe the results of analyses and trade studies that are to be 
completed to support the development of ROA Pilot Rating Criteria. The 
focus of this report will be on defining Pilot KSAs. The policy IPT will 
then use this information to develop recommendations for Pilot rating 
requirements. 
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d. Initial HSI Support for CCA, C3, CM, and Weather Technology 
Development (3/05; 9/05). This report will specify preliminary HSI 
requirements and design guideline information for identified technologies 
that support ROA flight in the NAS. The report will contain a literature 
review identifying HSI recommendations for program and HSI-identified 
technologies. Analytic, simulation, and flight test results will be documented 
as part of a process to downselect options for HSI technology interface 
requirements for the ROA system. 

e. Initial HSI Support for Simulation and Flight Test and Evaluation for 
CCA, C3, CM, and Weather (3/05; 9/05; other date(s) as required by 
test planning IPT). This report will specify preliminary requirements and 
design guideline information for support of simulation and flight test 
activities. Experiment objectives, methodologies, HSI data collection 
devices, and statistical results sections for each simulation or flight test 
study will be provided as part of the overall test plan document. As needed, 
the report also will contain mission scenarios and events to be included in 
the evaluations. 

7.1.3 2006 HSI Deliverables - 

a. Updated HSI Functional Requirements for ROA Flight in the NAS (9/06). 
This activity consists of identifying the major functions that the ROA vehicle 
systems and aircrew must perform to accomplish mission objectives. The report 
will identify primary functions and decomposed elements in the following 
categories: Crew Subfunctions, System Functional Requirements, and Preliminary 
Crew Control and Information Requirements. 

b. Updated HSI Requirements and Guidelines for Experimental Certification 
(9/06). This document will provide updated requirements for the design, 
development and certification of ROA AVCS. The report will focus on 
developing design guidelines for the interface between the ROA pilot and ATC, 
and between the pilot and the ROA. This document also describes methodologies 
that can be used to achieve AVCS certification. This report will also describe 
certification criteria, issues and validation concepts. It is also planned to update 
this report at the end of each step. 

c. Updated ROA Pilot Knowledge, Skill and Abilities (KSA) Updated for 
Medical and Training Requirements (2/06). This report will update the 
results of analyses and trade studies that are to be completed to support the 
development of ROA Pilot/Operator Crew Rating Criteria. This will include 
defining minimum qualifications for these pilots (experience, education, 
training, etc.) and the number of pilots required to operate an ROA. 

d. HSI Requirements for ATC Integration with ROAs (2/06). This report will 
update results of analytic evaluations that define the impact on air traffic 
controller human interfaces of potential ROA operations. It will include 
descriptions of FAA Order 71 10.65, air traffic controller functions, tasks, 
information and control requirements, and workload that will be issues for 
ROA operation. 


The following document was prepared by a collaborative team through the noted work package. This 
was a funded effort under the Access 5 Project. 

20 




e. Updated HSI Support for CCA, C3, CM, and Weather Technology 
Development (2/06; 9/06). This report will specify preliminary HSI 
requirements and design guideline information for identified technologies that 
support ROA flight in the NAS. The report will contain a literature review 
identifying HSI recommendations for program and HSI-identified 
technologies. Analytic, simulation, and flight test results will be documented as 
part of a process to downselect options for HSI technology interface 
requirements for the ROA system. 

f. Updated HSI Support for Simulation and Flight Test and Evaluation for 

CCA, C3, CM, and Weather (2/06; 9/06). This report will specify 
preliminary requirements and design guideline information for support of 
simulation and flight test activities. Experiment objectives, methodologies, 

HSI data collection devices, and statistical results sections for each simulation 
or flight test study will be provided as part of the overall test plan document. 

As needed, the report also will contain mission scenarios and events to be 
included in the evaluations. 


7.2 HSI SCHEDULE 

Figure 7-1 presents overall program schedule for the accomplishment of required 
Step 1 HSI analysis, design and T&E activities. 


HSI Deliverable 

FY04 Date 

FY05 Date 

FY06 Date 

Step 1 HSI Program Plan 

6/04 



Preliminary HSI Concept Requirements Definition 

9/04 



Preliminary HSI Design Guide 

9/04 



2005 




Initial HSI Functional Requirements for ROA Flight 
in the NAS 


9/05 


Initial HSI Requirements and Guidelines for 
Experimental Certification 


3/05; 9/05 


Initial HSI Requirements for Pilot Rating 


3/05; 9/05 


Initial HSI Support for CCA, C3, CM, and Weather 


3/05; 9/05 
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HSI Deliverable 

FY04 Date 

FY05 Date 

FY06 Date 

Technology Development 




Initial HSI Support for Simulation and Flight Test 
and Evaluation for CCA, C3, CM, and Weather 


3/03; 9/05; 
other date(s) 
as required 
by test 
planning 
IPT 


2006 




Updated HSI Functional Requirements for ROA 
Flight in the NAS 



9/06 

Updated HSI Requirements and Guidelines for 
Experimental Certification 



9/06 

ROA Pilot Knowledge, Skill and Abilities (KSA) 
updated for medical and training requirements 



2/06 

Updated HSI Requirements for ATC Integration 
with ROAs 



2/06 

Updated HSI Support for CCA, C3, CM, and 
Weather Technology Development 



2/06; 9/06 

Updated HSI Support for Simulation and Flight Test 
and Evaluation for CCA, C3, CM, and Weather 



3/06; 9/06; 
other date(s) 
as required by 
test planning 
IPT 


Figure 7-1: Schedule for the HSI Work Package for Step 1 

Appendix A 

Human Systems Integration (HSI) 
Mission and Function Analysis Data Forms 
May, 2004 
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Mission Analysis 


Please provide a narrative description of a representative flight of a UAV system with 
which you have the most experience. Describe the purpose of the flight. Provide a 
horizontal profile (e.g., footprint) and a vertical profile timeline showing numerous 
events. Briefing describe the most significant events for taxi out, departure, approach and 
landing; provide more detail for enroute portion of the flight - above FL400. After 
describing flight events, please identify what you consider to be the issues relative to this 
mission (e.g., speed, endurance, crew complement factors, and weather avoidance) that 
would have an impact on operation in the National Airspace System? 


Function Analysis 

Instructions: Please review the following list of air vehicle functions. Check the 
functions that you consider necessary to operate the UAV in a file and fly system in order 
to perfonn the intended flight described in the mission analysis. If a particular function is 
not available describe the expected impact to operations. 


ID 

Functionality Description 

Functional Capability 



FLIGHT 1 


PITCH/ROLL/YAW 




TRUE AIRSPEED 




GROUND SPEED 




INDICATED/EQUIVALENT AIRSPEED 




MACH 




ALTITUDE 




BAROMETRIC SETTING 




HEIGHT ABOVE GROUND 




VERTICAL SPEED 




FLIGHT PATH ANGLE 




BANK ANGLE LIMIT 



COMMUNICATIONS 


OPERATOR- ATC COMMUNICATIONS 




OPERATOR-UAV COMMUNICATIONS 




1 CONTROL AND/OR DISPLAY 
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OPERATOR- VHF/UHF COMMUNICATIONS 




UAV-ATC COMMUNICATIONS 



NAVIGATION 1 


HEADING 




HEADING INTENT 2 




TRACK (MAG/TRUE) 




TRACK INTENT (MAG/TRUE ) 2 




VERTICAL PROFILE GUIDANCE 




VERTICAL PROFILE DEVIATION 




LATERAL PROFILE GUIDANCE 




LATERAL PROFILE DEVIATION 




TRUE AIRSPEED INTENT 2 




GROUND SPEED INTENT 2 




INDICATED/EQUIVALENT AIRSPEED 
INTENT 2 




MACH INTENT 2 




ALTITUDE INTENT 2 




VERTICAL SPEED INTENT 2 




INTERACTIVE FLIGHT PLAN REVISING 




FLIGHT PATH ANGLE INTENT 2 




RNAV 




RADIO (VHF/UHF) 




DME DISTANCE 




PRECISION APPROACH 




NON-PRECISION APPROACH 




AUTOMATIC DIRECTION FINDING 
(VHF/UHF) 




MARKER BEACON IDENTIFICATION 




GNSS (ENROUTE) 




FANS 




DISTANCE TO GO 




TIME TO GO 




WIND SPEED AND DIRECTION 




2 INTENT INFORMATION REQUIRED BY SOME FUTURE CNS/ATM CONCEPTS 
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ETA AT WAYPOINT 




ACTUAL NAVIGATION PERFORMANCE/ 
REQUIRED NAVIGATION PERFORMANCE 




AIRCRAFT POSITION 




NAVIGATION PERFORMANCE 
PREDICTION 




GROUND MAPPING RADAR 




IDENTIFY GROUND TRAFFIC (DIRECT 
VIEW) 3 




IDENTIFY GROUND TRAFFIC 
(ELECTRONIC) 4 




FOLLOW GROUND TRAFFIC (DIRECT 
VIEW) 




FOLLOW GROUND TRAFFIC 
(ELECTRONIC) 




IDENTIFY HOLD LINES 




IDENTIFY IN FLIGHT TRAFFIC (DIRECT 
VIEW) 




IDENTIFY IN FLIGHT TRAFFIC 
(ELECTRONIC) 




FOLLOW IN FLIGHT TRAFFIC (DIRECT 
VIEW) 




FOLLOW IN FLIGHT TRAFFIC 
(ELECTRONIC) 



SURVEILLANCE 


PERMIT ATC SSR PARAMETER 
IDENTIFICATION 



HAZARD AVOIDANCE - SURVEILLANCE 3 


WEATHER (DIRECT VIEW) 




WEATHER (RADAR) 




PRECIPITATION (DIRECT VIEW) 




PRECIPITATION (ELECTRONIC) 




3 DIRECT VIEW = VISUAL OBSERVATION AND/OR VIDEO PICTURE OF TRAFFIC 

4 ELECTRONIC = DATA DISPLAYED TO OPERATOR THAT PROVIDE POSITION INFORMATION 
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TURBULENCE (DIRECT VIEW) 




TURBULENCE (ELECTRONIC) 




LIGHTNING (DIRECT VIEW) 




LIGHTNING (ELECTRONIC) 




VOLCANIC ASH (DIRECT VIEW) 




WINDSHEAR 




TERRAIN (DIRECT VIEW) 




TERRAIN (ELECTRONIC) 




TRAFFIC - GROUND (DIRECT VIEW) 




TRAFFIC - GROUND [ELECTRONIC 
(XPONDER; DL) 




TRAFFIC - IN FLIGHT (DIRECT VIEW) 




TRAFFIC - IN FLIGHT (ELECTRONIC) 




DISPLAY OF TRAFFIC INFORMATION 




FORMATION FLIGHT 




INSPECT/CLEAR TAXIWAY/RUNWAY 




INSPECT/CLEAR AREA PRIOR TO DEPARTURE 




INSPECT/CLEAR PRIOR TO ARRIVAL 



HAZARD AVOIDANCE - VEHICLE SYSTEMS 5 


STRUCTURAL ICING avoidance 




ENGINE ICING 




IMAGING SYSTEM ICING/FOGGING 




EXTENDED ELECTRICAL POWER DURATION 



MISCELLANEOUS 


IDENTIFY VFR CONDITIONS 




What about Health and Status Information, access to 
BIT, Sensor/system controls, etc.? 




Information Requirements 

What information do you not have that he would like to have for the flight described in 
mission analysis? What does limitation of this information mean for complying with 

5 EQUIVALENCY TO CURRENT VEHICLES 
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ATC instructions and other operational limitations? Please list and give examples as 
possible. Again, might be worthwhile to provide structure- Break out by specific 
phases/tasks. What tasks are the hardest? Why are they difficult? What can be done to 
reduce difficulty? 

Lessons-Learned 

What are some of the major hurdles that have to be overcome? What workstation HSI 
deficiencies do you presently have that impacts workload or situation awareness (SA), 
e.g., what additional info should be provided, how could controls be redesigned, more 
automation, better awareness of what automation is doing, ergonomic concerns, etc.? 
What can be done to reduce workload/enhance SA? Again, probably better to provide 
more structure - For example! 

• HSI deficiencies (Controls and Displays) - Please list the top 5/10 deficiencies in 
your current work station. This includes the adequacy of information provided, 
the adequacy of control operations, etc. 

• Automation Concerns - Please any deficiencies in current automation schemes. If 
you identify any concerns what can be done to improve operability (e.g., better 
feedback on state of automation, intent of automation, automation failures, etc.) 

• HSI Ergonomic Deficiencies - Please list any workstation or environment 
concerns that could impact performance, or operator comfort/safety. 
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